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Abstract 

Objective To develop and validate risk algorithms (QBIeed) for estimating 
the absolute risk of upper gastrointestinal and intracranial bleed for 
patients with and without anticoagulation aged 21-99 years in primary 
care. 

Design Open cohort study using routinely collected data from general 
practice linked to hospital episode statistics data and mortality data 
during the five year study period between 1 January 2008 and 1 October 
2013. 

Setting 565 general practices in England contributing to the national 
QResearch database to develop the algorithm and 188 different 
QResearch practices to validate the algorithm. All 753 general practices 
had data linked to hospital episode statistics and mortality data at 
individual patient level. 

Endpoint Gastrointestinal bleed and intracranial bleed recorded on 
either the linked mortality data or the linked hospital records. 

Participants We studied 4.4 million patients in the derivation cohort with 
16.4 million person years of follow-up. During follow-up, 21 641 patients 
had an incident upper gastrointestinal bleed and 9040 had an intracranial 
bleed. For the validation cohort, we identified 1 .4 million patients 
contributing over 4.9 million person years of follow-up. During follow-up, 
6600 patients had an incident gastrointestinal bleed and 2820 had an 
intracranial bleed. We excluded patients without a valid Townsend score 
for deprivation and those prescribed anticoagulants in the 180 days 
before study entry. 

Risk factors Candidate variables recorded on the general practice 
computer system before entry to the cohort, including personal variables 
(age, sex, Townsend deprivation score, ethnicity), lifestyle variables 
(smoking, alcohol intake), chronic diseases, prescribed drugs, clinical 
values (body mass index, systolic blood pressure), and laboratory test 
results (haemoglobin, platelets). We also included previous bleed 
recorded before entry to the study. 



Results The final QBIeed algorithms incorporated 21 variables. When 
applied to the validation cohort, the algorithms in women explained 40% 
of the variation for upper gastrointestinal bleed and 58% for intracranial 
bleed. The corresponding D statistics were 1 .67 and 2.42. The receiver 
operating curve statistic values were 0.77 and 0.86. The sensitivity values 
for the top 1 0th of men and women at highest risk were 38% and 51 %, 
respectively. There were similar results for men. 

Conclusion The QBIeed algorithms provided valid measures of absolute 
risk of gastrointestinal and intracranial bleed in patients with and without 
anticoagulation as shown by the performance of the algorithms in a 
separate validation cohort. Further research is needed to evaluate the 
clinical outcomes and the cost effectiveness of using these algorithms 
in primary care. 

Introduction 

Anticoagulants are used in the prevention and treatment of 
venous thromboembolism. They are also used to reduce risk of 
ischaemic stroke, 1 especially among patients with atrial 
fibrillation. The use of anticoagulants is likely to increase in 
future, especially since guidelines from the UK National Institute 
for Health and Care Excellence encourage more systematic 
identification of patients at high risk of venous 
thromboembolism or stroke who might benefit from 
anticoagulation. For example, in 2010 NICE issued new 
guidance to improve the prevention of venous thromboembolism 
for patients, using cost effective interventions. 2 In January 2014 
NICE issued draft guidance on the management of atrial 
fibrillation, which included assessment of the risks of stroke as 
well as the risks and benefits associated with anticoagulation. 3 

New tools now exist to quantify the absolute risk of thrombosis 4 
as well as those at risk of ischaemic stroke in primary care, 
including patients with atrial fibrillation. 5 Although the risk of 
stroke in patients with atrial fibrillation can be reduced by 
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anticoagulation, 1 many patients with atrial fibrillation are not 
currently prescribed anticoagulation even though it is 
incentivised in the general practice Quality and Outcomes 
Framework. 6 This may reflect concerns about monitoring, or 
uncertainties about the potential balance of risks and benefits 
for an individual patient, including the potential adverse 
haemorrhagic effects of traditional anticoagulants such as 
warfarin. Novel oral anticoagulants (factor Xa inhibitors and 
direct thrombin inhibitors) exist that have the advantage of not 
requiring regular international normalisation ratio (INR) blood 
test monitoring. 7 Data on safety and efficacy of the novel 
anticoagulants is still accumulating. However, a recent 
meta-analysis of three randomised controlled trials 8 reported 
that the new anticoagulants are more effective at reducing all 
cause stroke and systemic embolism (relative risk 0.78, 95% 
confidence interval 0.67 to 0.92) in people with atrial fibrillation 
compared with warfarin. However, data on risks of major 
bleeding (0.88, 0.71 to 1.09) were inconclusive, 8 with a 
suggestion of an increased risk of gastrointestinal bleeding ( 1 .25, 
0.91 to 1.72) compared with warfarin. 8 Quantifying the absolute 
risk of bleed for an individual receiving anticoagulation 
treatment is important because it can help clinicians identify 
patients for whom the benefit of anticoagulation outweighs the 
risk of bleed, help inform the discussion with patients about the 
risks and benefits of anticoagulation, and allow clinicians to 
monitor more carefully patients at high risk to help reduce their 
risk. For example, this could be achieved by modifying known 
risk factors for bleed, including avoidance of the use of 
concurrent drugs known to increase the risk of bleed, such as 
non-steroidal anti-inflammatory drugs (NSAIDs) and 
antidepressants. 9 

Several clinical risk prediction rules have been developed to 
assess the risk of bleed. Among the first were the 
HEMORRHAGES, 10 ATRIA scores," and RIETE and OBRI 
scores. 12 More recently the HAS-BLED score 13 has been 
developed and is recommended in the European Society of 
Cardiology guidelines. 14 HAS-BLED has several advantages 
compared with previous scores as it does not require information 
unlikely to be available in clinical practice, such as genetic 
information. It has also has out-performed previous scores in 
identifying clinically relevant bleeding. 15 The populations used 
to develop HAS-BLED, as with HEMORRHAGES, 10 are 
unlikely to be representative of patients in UK primary care, 
having been recruited through an American insurance scheme 10 
or through secondary care as part of a clinical trial." 13 The 
studies are based on small cohorts of patients with low numbers 
of events. The resulting scores are implemented using simple 
counting systems, do not include some important risk factors 
for bleed, and do not give an absolute risk of bleed. Although 
including more risk factors within a risk score generally 
improves risk stratification, it also makes tools such as 
HAS-BLED increasingly cumbersome to use in everyday 
clinical practice. Limitations of the ATRIA scores have been 
highlighted, 16 including use of a cohort that had already received 
anticoagulants, exclusion of patients with risk factors, and 
counterintuitive results, including an apparent protective effect 
of previous bleed and concurrent use of antiplatelet treatment. 16 

We developed and validated a new set of risk prediction 
algorithms to predict risk of gastrointestinal and intracranial 
bleed in new users of anticoagulants (rather than existing users). 
We wanted to develop algorithms that quantify absolute risk of 
bleed, which can be communicated to patients to aid decision 
making in the consultation, automatically populated from the 
patients record and integrated into general practice computer 



systems, and updated readily over time, as the pattern of 
prescribing of both anticoagulant drug classes may change. 

Methods 

Study design and data source 

We conducted a cohort study of a large primary care population 
using version 38 of the QResearch database (www.qresearch. 
org). QResearch is a large validated primary care electronic 
database containing the health records of 14 million patients 
registered from 753 general practices using the Egton Medical 
Information System (EMIS) computer system. 17 Practices and 
patients contained on the database are nationally representative 18 
and similar to those on other primary care databases using other 
clinical software systems." We included all QResearch practices 
in England once they had been using their current EMIS system 
for at least a year (to ensure completeness of recording of 
morbidity and prescribing data), randomly allocating three 
quarters of practices to the derivation dataset and one quarter 
to the validation dataset. 

Overall cohort selection 

We identified an open cohort of patients aged 21-99 years at 
the study entry date, drawn from patients registered with eligible 
practices between 1 January 2008 and 1 October 20 1 3 . We used 
an open, rather than closed, cohort design, as this allows patients 
to enter the population throughout the whole study period rather 
than require registration on 1 January 2008, thus better reflecting 
the realities of routine general practice. We excluded patients 
without a valid postcode related Townsend deprivation score, 
and existing users of anticoagulants by excluding those 
prescribed anticoagulants in the 180 days before their study 
entry date. To help minimise potential biases we used an incident 
user design. 20 

For each patient we determined an initial entry date to the cohort, 
which was the latest of the following dates: 21st birthday, date 
of registration with the practice plus one year, date on which 
the practice computer system was installed plus one year, and 
the beginning of the study period (1 January 2008). For patients 
who had started an anticoagulant after their initial entry date, 
we assigned an index entry date that was the date of their first 
prescription of the anticoagulant during the study period. 

We censored patients at the earliest date of the first upper 
gastrointestinal haemorrhage or intracranial haemorrhage, death, 
deregistration with the practice, last upload of computerised 
data, three months after their last prescription for anticoagulants 
(if applicable), five years after their study entry date, or the 
study end date (1 October 2013). For intermittent users of 
anticoagulants, if there was a gap of more than three months 
between the end of one prescription and the start of the next, 
we censored them after three months from the start of the gap 
even if the patient had future prescriptions. Patients who 
switched anticoagulants during the study period were censored 
one month after their last prescription before the switch. 

Outcomes 

Our two primary outcomes were incident gastrointestinal bleed 
and incident intracranial bleed occurring within five years of 
study entry. We used ICD-10 (international classification of 
diseases, 10th revision) codes to identify events that were 
recorded on either the linked hospital record or the mortality 
record. Supplementary table 1 lists the ICD-10 codes used for 
upper gastrointestinal bleeding and supplementary table 2 the 
codes used for intracranial bleed. Upper gastrointestinal bleed 
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was defined as any upper gastrointestinal ulcer with perforation, 
bleed, or both; melena; haematemesis; laceration with bleed; 
varices with bleed; haemorrhagic gastritis; and other unspecified 
gastrointestinal bleeds. Intracranial bleed was defined as 
subarachnoid haemorrhage, intracerebral haemorrhage, subdural 
or extradural haemorrhage, and other unspecified intracranial 
bleeds. 

The QResearch database is linked at individual patient level to 
both the hospital admissions data and the Office for National 
Statistics mortality records using a project specific 
pseudonymised National Health Service number. The recording 
of NHS numbers is valid and complete for 99.8% of patients 
with records on QResearch, 99.9% for Office for National 
Statistics mortality records, and 98.6% for hospital admissions 
records. 21 22 We defined patients as having the outcome of 
interest if there was a record of the relevant ICD-10 code in 
either their hospital record or their linked mortality record. For 
example, for a patient to be classified as having been admitted 
to hospital due to bleeding, the ICD-10 code would be recorded 
as the primary diagnosis in the hospital admissions data. 
Similarly for a patient to have been considered to have died due 
to bleeding, the relevant ICD-10 code would be recorded as 
either the primary or the underlying cause of death in the Office 
for National Statistics mortality record. We did not include 
events that were recorded on the general practice record but not 
on either the hospital or mortality record as we wanted to ensure 
the events were likely to represent severe bleeding. 

We separately quantified the risk of upper gastrointestinal bleed 
and intracranial bleed, as has been done in relevant clinical 
trials. 8 These two events are potentially life threatening or life 
changing and may be preventable. We initially considered 
having a more general outcome of "any haemorrhagic event" 
but decided against this given the disparate nature of the events, 
the rarity with which some occur, the lack of relevant ICD-10 
codes for some outcomes, and the difficulty of applying 
definitions requiring clinical adjudication to routinely collected 
data. 

Exposure 

The primary exposure of interest was use of anticoagulants, 
including warfarin, phenindione, dabigatran, rivaroxaban, or 
apixaban. We identified new users of anticoagulants during the 
study period by identifying those with at least one prescription 
issued after the date of initial entry to the study. We excluded 
patients who had been prescribed one or more prescriptions in 
the past 180 days before their study entry date. The remaining 
patients were classified as non-users of anticoagulants. 

Predictor variables for consideration 

Using similar definitions where relevant we compiled a list of 
risk factors (box) based on the September 2013 edition of the 
British National Formulary, literature, and existing risk 
prediction scores. 

For clinical values, we used the most recently recorded before 
entry to the cohort (box). For new users of anticoagulants, we 
used the values recorded before they started anticoagulants. We 
identified medical conditions based on the patient's primary 
care record, using standard definitions of Read codes where 
available. We considered the conditions to be present if they 
were entered in the patients' record before entry to the cohort. 

We identified new users of anticoagulants and users of other 
drugs thought to increase the risk of bleeding or that interact 
with anticoagulants (box). To qualify for being exposed to the 
relevant drug at study entry, participants had to have at least 



two prescriptions, with the most recent one within the preceding 
28 days before entry to the cohort. 

Derivation of the models 

We used published methods to derive and validate models that 
could be applied to new users of anticoagulants. 23 26 As in 
previous similar studies, we decided a priori to derive separate 
models in men and women since the predictors and their 
weightings are likely to be different. We therefore derived four 
models in total — gastrointestinal bleed and intracranial bleed 
for men and a corresponding set for women. We used multiple 
imputation to replace missing values for body mass index, 
systolic blood pressure, smoking, and alcohol intake, and used 
these values in our main analyses. 27 30 We carried out five 
imputations. We used Cox's proportional hazards models to 
estimate the coefficients for each risk factor for men and women 
separately, using Rubin' s rules to combine the results across 
the imputed datasets. We used fractional polynomials to model 
non-linear risk relations with continuous variables. 3 ' We used 
backward selection procedures to eliminate variables, using a 
P value of 0.01 to obtain parsimonious models. We focused on 
predictor variables that have a clinical face validity and are 
included in similar scores and those that have a hazard ratio of 
<0.85 or >1. 15 (for categorical exposures), selecting those that 
met these criteria for inclusion in the final model. 

We examined interactions between predictor variables and age. 
We also tested for interactions between anticoagulants and drugs 
thought to increase the risk of a bleed, such as antiplatelets and 
other drugs listed previously. We included significant interaction 
terms in the final models where these were plausible. 

We took the regression coefficients for each variable from the 
final models and used these as weights, which we combined 
with the baseline survivor function for each outcome evaluated 
at each of 1-5 years to derive absolute risk equations at each 
years' follow-up. 

Validation of the models 

We applied each of the models obtained from the derivation 
cohort to the validation cohort and calculated predicted risks 
for each patient and measures of discrimination (D statistic 32 
and area under the receiver operating characteristic curve 
statistic) and calibration (comparing observed risk at five years 
with mean predicted risks by 10th of predicted risk), and 
displayed these as plots. Discrimination is the ability of the risk 
prediction model to differentiate between patients who do and 
do not experience an event during the study. This can be 
quantified by calculating the area under the receiver operating 
characteristic curve statistic; where a value of 1 represents 
perfect discrimination. The D statistic 32 is also a measure of 
discrimination that is specific to censored survival data. As with 
the receiver operating characteristic curve, higher values indicate 
better discrimination. We also calculated an R 2 measure specific 
to censored survival data — it measures explained variation, and 
higher values indicate more variation explained. 33 

We used the predicted risks in the QResearch validation cohort 
to define a threshold for the top 10% of patients at highest 
predicted risk of bleed. We carried out a separate analysis 
restricted to patients in the validation cohort with atrial 
fibrillation. We used all the available data on the QResearch 
database to maximise the power and also the generalisability 
of the results. We used STATA (version 13) for all analyses. 
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Risk factors associated with risk of bleeding 

Personal and life style 

• Age (years) at study entry 23 

■ Self assigned ethnicity: white or not recorded, Indian, Pakistani, Bangladeshi, other Asian, Black African, Black Caribbean, Chinese, other (including 
mixed) 

• Smoking status: non-smoker, former smoker, light smoker (<10 cigarettes/day), moderate smoker (10-19 cigarettes/day), heavy smoker (>20 
cigarettes/day) 

• Deprivation: Townsend score 23 24 

• Alcohol intake (units/day): none, <1 , 1 or 2, 3-6, 7-9, >9 13 

Clinical values (most recent before study entry) 

■ Body mass index 

• Systolic blood pressure 

• Haemoglobin <1 1 0 g/L 

• High or low platelet count (<150/ul_ or >480/ul_) 

Medical conditions thought to increase the risk of bleed 

• Treated hypertension' 3 : (recorded diagnosis of hypertension and a current prescription for at least one antihypertensive) 

■ Atrial fibrillation or flutter (including paroxysmal atrial fibrillation) 

• Chronic renal disease 13 : Read code indicating glomerular filtration rate <60 mL/min or recorded diagnosis of major renal disease, including renal 
transplant or dialysis 

• Chronic liver disease or chronic pancreatitis' 3 : including cirrhosis, liver failure, liver fibrosis, portal hypertension, chronic hepatitis, autoimmune hepatitis, 
and chronic pancreatitis 

■ Type 1 or type 2 diabetes (including diagnosis of diabetes or treatment with oral hypoglycaemic agents or insulin) 

■ Ischaemic stroke or transient ischaemic attack 13 

• Congestive cardiac failure (including left ventricular failure, right heart failure, pulmonary oedema) 

• Venous thromboembolism (including pulmonary embolism or deep vein thrombosis) 

• Cancer (any diagnosis of cancer, excluding non-melanoma skin cancer) 

■ Peptic ulcer (including gastric, oesophageal, or duodenal ulceration) 

• Oesophageal varices or arteriovenous malformation 

■ Previous bleed recorded in the general practice record (including intracranial, upper gastrointestinal, haematuria, or haemoptysis) 

Drugs thought to increase bleeding risk or interact with anticoagulants 

• Anticoagulants (new users) 

• Antiplatelets: aspirin, dipyramidole, clopidogrel 13 

■ Non-steroidal anti-inflammatory drugs 

■ Corticosteroids 

• Statins 

■ Antidepressants 3 

■ Rifampicin 

• Carbamazepine 

• Phenytoin 

• Quinidine 

• Verapamil 

■ Clarithromycin 

• Amiodarone 

• Chloramphenicol 

• Antifungals 



Results 

Practices and patients 

Overall, 753 practices in England met our inclusion criteria and 
all had been using their current computer system for at least one 
year. Of these, 565 were randomly assigned to the derivation 
dataset and 188 to the validation dataset. We identified 4 437 
419 patients aged 21-99 years in the derivation cohort. Of these, 
7870 (0.2%) had a missing Townsend score and 50 698 (1.2%) 
had at least one prescription for anticoagulants in the 180 days 
before study entry, leaving 4 378 851 eligible patients for 
analysis. We identified 1 373 253 patients aged 21-99 years in 
the validation cohort. Of these, 2041 (0.1%) had a missing 
Townsend score and 15 022(1.1%) had at least one prescription 



for anticoagulants in the 180 days before their study entry date, 
leaving 1 358 231 eligible patients for analysis. 

Table 1 U shows the characteristics of eligible patients prescribed 
warfarin, novel anticoagulants, and neither drug for both the 
derivation and validation cohort recorded before entry to the 
cohort. Of the 52 485 patients in the derivation cohort prescribed 
warfarin, 23% had a previous bleed. Of the 1641 patients 
prescribed novel anticoagulants, 27% had a previous bleed. 
Overall, 27% of patients prescribed warfarin had concurrent 
prescriptions for non-steroidal anti-inflammatory drugs, 11% 
for corticosteroids, and 37% for antiplatelets. The corresponding 
figures for patients prescribed novel anticoagulants were 39%, 
10%, and 35%. Although the validation cohort was drawn from 
an independent group of practices, the baseline characteristics 
were generally similar to those for the derivation cohort. 
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Incidence rates of bleed 

Table 2JJ. shows the incidence of both types of bleed during 
follow-up in patients prescribed warfarin, new anticoagulants, 
or neither drug. 

Overall in the derivation cohort, we identified 21 641 patients 
with an incident upper gastrointestinal bleed and 9040 with an 
intracranial bleed, arising from 16.4 million person years of 
observation. The crude incidence rate for upper gastrointestinal 
bleed was 1.32 per 1000 person years. The crude incidence rate 
for intracranial bleeds was 0.55 per 1000 person years. 

We standardised incidence rates to the overall age distribution 
of the derivation cohort. The age standardised incidence rate of 
upper gastrointestinal bleed per 1000 person years was 1.34 
(95% confidence interval 1.32 to 1.36) in patients not prescribed 
anticoagulants, 5.83 (4.33 to 7.33) in those prescribed warfarin, 
and 2.69 (0.36 to 5.03) in those prescribed new anticoagulants. 
The corresponding figures for intracranial bleed per 1000 person 
years were 0.55 (0.54 to 0.56), 1.89 (1.46 to 2.32), and 1.49 
(1.27 to 2.86). 

Model development 

Upper gastrointestinal bleed 

Table 30 shows the models for upper gastrointestinal bleed in 
men and women. Details of the fractional polynomial terms for 
age and body mass index are given in the footnote of table 3 
and in figure 1 |J . The final model included 20 variables in 
addition to use of anticoagulants. These were age, body mass 
index, Townsend score, smoking status, ethnicity, alcohol intake, 
previous bleed, oesophageal varices, chronic liver disease or 
pancreatitis, atrial fibrillation, venous thromboembolism, 
congestive cardiac failure, treated hypertension, cancer, recent 
abnormal platelet counts (<150/uL or >480/uL), and current 
prescriptions for antiplatelets, non-steroidal anti-inflammatory 
drugs, corticosteroids, antidepressants, and anticonvulsants 
(phenytoin or carbamazepine). 

At the mean age, there was a fourfold increased risk of upper 
gastrointestinal bleed associated with new use of anticoagulants 
in men (adjusted hazard ratio 4.43, 95% confidence interval 
3.32 to 5.92) and in women (3.89, 2.75 to 5.49). The risk of 
upper gastrointestinal bleed increased with age in both sexes 
(fig 1). Two interactions were included in the final model, which 
were between the age terms and anticoagulant use and between 
age and previous bleed. Both of these indicated higher hazard 
ratios for anticoagulant use and previous bleed among younger 
patients compared with older patients (fig 1). 

In terms of concurrent prescribed drugs, the risks associated 
with antidepressants were higher than those associated with 
non-steroidal anti-inflammatory drugs and antiplatelets. In 
women, the adjusted hazard ratio for antidepressants was 1.57 
(95% confidence interval 1.51 to 1.64) and for non-steroidal 
anti-inflammatory drugs was 1.16 (1.11 to 1.21). There was a 
suggestion of a dose response for both alcohol and smoking. 
Heavy and moderate smokers had higher risks than non-smokers 
and former smokers. There was a "U shaped" effect for alcohol 
intake, with lower risks associated with trivial, light, and 
moderate alcohol consumption compared with none, and higher 
risks associated with heavy and very heavy consumption. 

Intracranial bleed 

Table 4JJ. shows the adjusted hazard ratios and 95% confidence 
intervals for the final model for intracranial bleed. The final 
model included 15 variables in addition to use of anticoagulants, 
which were age, body mass index, Townsend deprivation score, 



smoking status, ethnicity, alcohol intake, previous bleed, 
oesophageal varices, chronic liver disease or pancreatitis, atrial 
fibrillation, treated hypertension, recent abnormal platelet counts 
(<150/uL or >480/uL), and current prescriptions for antiplatelets, 
antidepressants, and anticonvulsants (phenytoin or 
carbamazepine). The footnote of table 4 gives details of the 
fractional polynomial terms for age and body mass index. One 
interaction terms were included in the final model, which were 
between the age terms and anticoagulant use. 

At the mean age, there was around a fourfold increased risk of 
intracranial bleed associated with new use of anticoagulants in 
men (adjusted hazard ratio 3.99, 95% confidence interval 1.86 
to 8.55) and in women (3.62, 1.25 to 10.49). The risk of 
intracranial bleed increased with age in both sexes (fig 2JJ.). The 
age interaction terms indicated that middle aged men and women 
tended to have higher hazard ratios associated with 
anticoagulation compared with younger and older patients (fig 
2). 

Calibration and discrimination in the 
validation cohort 

For the validation cohort, we identified nearly 1 .4 million 
patients contributing over 4.9 million person years of follow-up. 
During follow-up, 6600 patients had an incident upper 
gastrointestinal bleed and 2820 had an intracranial bleed. Table 
5JJ. shows the performance of each model in the validation 
cohort. 

For the upper gastrointestinal bleed outcome, the algorithms 
explained approximately 41% of the variation in women and 
37% of the variation in men. The D statistic was 1 .70 in women 
and 1.57 in men. The receiver operating characteristic curve 
value was 0.77 for women and 0.75 for men. For intracranial 
bleed, the algorithms explained approximately 58% of the 
variation in women and 53% of the variation in men. The D 
statistic was 2.40 in women and 2.19 in men. The receiver 
operating characteristic curve value was 0.85 for women and 
0.81 for men. 

To assist in the comparison with other bleeding risk scores we 
repeated the validation restricting the sample to men and women 
with atrial fibrillation (n=12 560) and we used the intracranial 
bleed outcome, which is comparable to that used in validation 
analyses of other scores. 15 The receiver operating characteristic 
curve statistic for men and women combined was 0.65 (0.61 to 
0.69). 

Table 6 > shows the performance statistics of both algorithms 
in the full validation cohort for the 10% of men and women at 
highest predicted risk of each outcome. For example, for the 
10% of men and women at highest predicted risk of an 
intracranial bleed (that is, those with a five year predicted risk 
score of 0.7% or higher), the sensitivity was 51%, and the 
observed risk at five years was 1 .5%. For upper gastrointestinal 
bleed, the corresponding sensitivity was 38%, with an observed 
risk of 2.7% at five years. 

Figure 3ij displays the predicted and observed risks of bleed at 
five years across each 10th of predicted risk (1 representing the 
lowest risk and 10 the highest risk) for each of the four models. 
Overall the models were well calibrated. 

Figure 4 j shows two clinical examples of using the web 
calculator at www.qbleed.org to calculate the absolute risk of 
bleed with and without anticoagulants in individual patients. 
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Discussion 

We developed and validated two models (collectively known 
as the QBleed algorithms) to identify patients at high risk of 
gastrointestinal or intracranial bleed. The QBleed algorithms 
incorporate predictor variables that are associated with an 
increased risk of haemorrhagic events, including 
sociodemographic variables, lifestyle, morbidity, drugs, and 
laboratory test results such as abnormal platelet function. The 
algorithms can be applied to any adult in a primary care setting 
regardless of whether they have had a previous bleed and have 
the advantage of evaluating risk in new users of oral 
anticoagulants, which is likely to be the highest risk period for 
bleeding. 

The QBleed algorithms are intended to help inform decisions 
within the consultation about the risks and benefits of patients 
in primary care using anticoagulants. For example, the doctor 
can assess the patient's five year risk of stroke using QStroke 5 
or five year risk of thrombosis using QThrombosis 4 against the 
risk of bleed with or without anticoagulation using QBleed. The 
doctor and patient also can review factors that might ameliorate 
the risks, such as amending concurrent drugs. Overall this could 
help the doctor and patient assess whether the balance of risks 
and benefits is likely to be favourable or not given the patient's 
profile. 

The QBleed algorithms provide an estimate of absolute risk of 
two types of bleed for each year for each of the next five years. 
The algorithms are designed to be used before anticoagulant 
treatment is started using information already available at the 
time, rather than to guide the continuing use of anticoagulants, 
taking into account changes in treatment during follow-up. The 
algorithms to quantify risk of intracranial bleed performed better 
than the algorithms to predict upper gastrointestinal bleed. 

Comparison with other risk scores 

We reviewed other clinical risk prediction tools designed to 
help quantify risk of bleed among patients prescribed 
anticoagulants, including HEMORRHAGES, "' ATRIA, RIETE, 
OBRI,' 2 and HAS-BLED, 13 although these are all simple 
counting systems rather than tools to quantify absolute risk as 
in our study. They also predict a combined general "major 
bleed" outcome, which is not directly comparable to our study 
since they include events that cannot be identified from routinely 
collected data. The HAS-BLED alogorithm is a scoring system 
derived from a population of 3978 patients aged 18 or more 
with atrial fibrillation who had 53 haemorrhagic events during 
a one year follow-up period. 13 The score includes the variables: 
hepatic or renal disease; use of aspirin or non-steroidal 
anti-inflammatory drugs or alcohol; older age (>65 years); 
previous bleed; and uncontrolled hypertension, stroke, and labile 
international normalised ratio. The international normalised 
ratio test is taken while a patient is prescribed warfarin, therefore 
the HAS-BLED score is more applicable to patients with atrial 
fibrillation already prescribed warfarin than to new users of 
anticoagulants as in our study who may have additional 
indications for anticoagulation. One of the key problems with 
the HAS-BLED score is that the variables included in it nearly 
always scored parallel to the stroke risk (for example, CHADS2) 
score in patients with atrial fibrillation. This made it difficult 
to assess whether the risk of stroke or the risk of bleeding was 
higher. In contrast, QBleed and its associated risk scores 
(QStroke 5 and QThrombosis 4 for assessing stroke and thrombosis 
risk, respectively) use different variables, which are separately 
weighted and so are likely to give better assessment of the 
different risks. Also, HAS-BLED is a counting system resulting 



in a score between 0 and 9 rather than a measure of absolute 
risk of bleed as with QBleed. Although not directly comparable 
to HAS-BLED, the receiver operating characteristic curve value 
for QBleed (intracranial bleed) among patients with atrial 
fibrillation was 0.65 (95% confidence interval 0.61 to 0.69) in 
our validation cohort, which is slightly higher than the median 
C statistic from six cohort studies for HAS-BLED (0.61, 0.59 
to 0.62). 3 

HEMORRHAGES is another scoring system, which was 
derived from a cohort of 3791 elderly patients aged 65-95 years 
with atrial fibrillation participating in an American insurance 
scheme. 10 The study was small and included only 162 
haemorrhagic events during follow-up. The score ranges from 
0 to 12 and takes account of hepatic or renal disease, alcohol 
misuse, cancer, older age (>75 years), previous bleed, reduced 
platelet count, hypertension, anaemia, excessive risk of falls, 
stroke, and genetic polymorphism. We were unable to apply it 
to our dataset as QResearch does not include some of the 
relevant variables such as genetic information. A recent review 
by the National Institute for Health and Care Excellence reported 
that the median C statistic for HEMORRHAGES from four 
cohort studies was 0.61 (95% confidence interval 0.56 to 0.65). 3 
Although these studies are not directly comparable to QBleed, 
the C statistics presented tended to be lower than the receiver 
operating characteristic curve value we report for QBleed 
(intracranial bleed) in patients with atrial fibrillation (0.65, 0.61 
to 0.69). 

Incidence rates and predictor variables 

The incidence rates of events and the predictor variables 
included in our final models have good face validity compared 
with the literature. For example, our crude rate of hospital 
admission or death for upper gastrointestinal haemorhage (1.32 
per 1000 person years) is similar to that reported by Button and 
colleagues in a Welsh study (1.34 per 1000 person years). 23 It 
is marginally higher than the rates reported by Crooks and 
colleagues 24 (0.87 per 1000 person years) and Rockall and 
colleagues 34 (1.03 per 1000 person years), both of which used 
slightly different definitions. Our crude incidence rate of 
intracranial bleed (which includes intracerebral, subarachnoid 
, intradural, and extradural bleed) was 0.55 per 1000 person 
years, which is consistent with the incidence of intracerebral 
and subarachnoid bleeds of 0.4 per 1000 person years reported 
by Nilsson in Sweden 35 and with the incidence rate for 
intracerebral bleeds of 0.25 per 1000 reported in a systematic 
review by Van Asch. 36 

We found that nearly all of the relevant available predictors 
included in HEMORRHAGES 10 and HAS-BLED 13 met the 
inclusion criteria for incorporation into our models. We found 
that increasing levels of material deprivation were associated 
with higher risk of bleed, including upper gastrointestinal bleed, 
supporting the findings of Button and colleagues. 23 Similar to 
Tata and colleagues, we found that antidepressants were 
independently associated with an increased risk of upper 
gastrointestinal bleed.' For some variables, which were tested 
but not included in our final model (such as rifampicin and 
quinidine), there were too few patients exposed and too few 
events to allow a robust analysis. We decided to include several 
variables in the models (such as peptic ulcer and oesophageal 
varices), although they have now been identified as a 
contraindication to the new anticoagulants. This information 
was published after the end of the study period and does not yet 
apply to warfarin. One of the advantages of developing risk 
tools using QResearch is that the risk model can be updated 
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over time should new predictors be identified or existing 
predictors lose relevance. 

Thresholds 

We have not provided definite comment on what threshold of 
absolute risk should be used for intervention or avoiding 
anticoagulants as that would require consideration of the balance 
of risk and benefit for an individual and their choice and require 
cost-effectiveness analyses, which are outside the scope of this 
study. We have, however, provided analyses using the top 10% 
of absolute risk as the threshold of risk, which can be used to 
help inform future analyses. Sensitivity is important as it is a 
measure of how well the algorithm performs in finding cases 
that might be suitable for intervention. If the risk threshold is 
set too high, then the sensitivity will be low and a large number 
of patients with bleeds will be "missed" by the algorithm. 
Conversely, a high risk threshold is likely to result in a higher 
positive predictive value, which means a higher proportion of 
those identified are likely to go on to have a bleed. So, at the 
population level there is a balance to be struck between the 
sensitivity and positive predictive value of the score, which 
depends on the risk threshold selected, resources available, and 
likely risks and benefits of the interventions. 

At the individual level, we envisage that the doctor and patient 
will review the absolute risk of intracranial or upper 
gastrointestinal bleed with and without anticoagulation. They 
can then compare this against the patient' s baseline risk of 
having a stroke or thrombosis and potentially the expected 
benefit to be derived from starting anticoagulation (although 
this is outside the scope of the current paper). They can then 
use this information to balance the estimated absolute 
individualised risks and benefits to inform the decision about 
anticoagulants. 

Comparison between warfarin and new 
anticoagulants 

We were interested to be able to quantify risk separately for 
new use of warfarin compared with new use of other 
anticoagulants (dabigatran, rivaroxaban, apixaban, or 
phenindione). However, it was apparent from initial analyses 
that the numbers of patients prescribed these new drugs are as 
yet insufficient as are the numbers of events over time to 
quantify the effects as separate variables with the risk model. 
We were able to compare the baseline characteristics of patients 
prescribed each category of drug (table 1) and to calculate age 
standardised incidence rates of events for our main outcomes 
separately for warfarin compared with the novel anticoagulants 
(table 2). The age standardised incidence rates of both types of 
bleed were higher with warfarin than with the new 
anticoagulants, but the confidence intervals overlapped. In light 
of the announcement by the US Food and Drug Administration 
on 30 December 20 13, 37 there is a need to further investigate 
potential safety issues with the new anticoagulants once more 
data are available. We plan to update the QBleed score when 
sufficient data are available to undertake direct comparisons 
between the older and newer agents, at which point we will be 
able to decide whether they should be presented as separate or 
combined variables. 

Methodological considerations 

The methods to derive and validate this model are the same as 
for a range of other clinical risk prediction tools derived from 
the QResearch database. 4 25 26 38 39 The strengths and limitations 
of the approach have already been discussed in detail, 4 19 38 40 42 



including information on multiple imputation of missing data. 
In summary, key strengths include size, duration of follow-up, 
and representativeness, and lack of selection, recall, and 
respondent bias. UK general practices have good levels of 
accuracy and completeness in recording clinical diagnoses and 
prescribed drugs. 43 We think our study has good face validity 
since it was conducted in the setting where most patients in the 
United Kingdom are assessed, treated, and followed up. Our 
database has linked hospital and mortality records for nearly all 
patients and is therefore likely to have picked up the majority 
of severe bleeds requiring admission to hospital or resulting in 
death thereby minimising ascertainment bias. We excluded 
patients without a valid deprivation score since this group may 
represent a more transient population where follow-up could 
be unreliable or unrepresentative. Their deprivation scores are 
unlikely to be missing at random so we did not think it would 
be appropriate to impute them. 

We used an incident user design to help minimise the impact 
of the biases that can arise from non-randomised studies such 
as in the current study. 20 In particular it can help reduce biases 
that occur when comparisons include patients who were already 
using the drug at the start of the study. An incident user design 
also helps to ensure that the analysis captures all events that 
occurred after the start of treatment and reduces the chance of 
bias that could otherwise occur from patients who stop 
treatments as a result of early side effects. 20 There is no 
consensus on the duration of the treatment-free period before 
study entry to classify a patient as a new user. Ray and 
colleagues defined incident use of non-steroidal 
anti-inflammatory drugs as patients who had not been prescribed 
treatment in the 365 days before entry to the cohort. 44 We 
selected 180 days to help ensure our methods were comparable 
to the recently published FDA protocol to assess safety of new 
anticoagulants. 37 

The present validation has been done on a completely separate 
set of practices and individuals to those used to develop the 
score, although the practices all use the same general practice 
clinical computer system (EMIS, the computer system used by 
55% of UK general practices). An independent validation study 
would be a more stringent test and should be done, but when 
such independent studies have examined other risk 
algorithms, 41 42 45 46 they have shown comparable performance 
compared with the validation in the QResearch database. 25 38 40 
We (or another academic team) intend to conduct a separate 
validation on an independent database using a different clinical 
system (Clinical Practice Research Datalink, CPRD) as part of 
a separate project. 

Other limitations of our study were that we measured exposure 
to anticoagulants based on prescriptions issued rather than a 
more direct measure of adherence to reflect whether the patient 
had actually taken the anticoagulants. However, we think it 
would be unusual for patients to continue to request repeat 
prescriptions for drugs that they are not taking. Also we did not 
take account of the degree of anticoagulation achieved during 
follow-up by analysing measurements for international 
normalised ratios. This was because our intention was to develop 
a score that would inform the decision to start anticoagulants, 
which would be based on information already available to the 
doctor and the patient rather than information that could vary 
during follow-up. Other limitations of our study included the 
lack of formally adjudicated outcomes, information bias, and 
potential for bias due to missing data. Also we included only 
two severe haemorrhagic outcomes, but these account for most 
types of severe bleed likely to be associated with 
anticoagulation. We were unable to include some rarer 
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haemorrhagic outcomes since these either do not have ICD 
codes or are not routinely recorded in the datasets available to 
us. 

These QBleed models have been developed using data from 
general practices in England and include a postcode related 
deprivation score. The models are therefore not likely to be 
applicable for clinical use in international settings without some 
modification of the English specific risk factors, and validation 
in the setting in which they are intended to be used. We included 
the Townsend score in the QBleed models since there is a clear 
relation between risk of bleeding and deprivation, which is 
captured by this variable. If we omitted it, it would tend to 
underestimate bleeding risk in patients from deprived areas. 
The corresponding QStroke score (which is used to identify 
some of those at high risk of stroke who might benefit from 
anticoagulation) includes deprivation and so we considered it 
would be more consistent to include it in both scores. In terms 
of international use of QBleed, where the Townsend score is 
not available, a locally relevant deprivation score could be 
constructed or adapted for a particular country to have a range 
between -11 (affluent) and 7 (deprived) to correspond to the 
range of Townsend scores that could be used instead (subject 
to local validation). 

Conclusion 

We developed and validated a set of new algorithms to predict 
absolute risk of upper gastrointestinal and intracranial bleed in 
new users of anticoagulants. QBleed has some advantages 
compared with current risk scoring methods. QBleed also 
provides an accurate measure of absolute risk of bleed in the 
general population as shown by its performance in a separate 
validation cohort. Further research is needed to evaluate the 
clinical outcomes and cost effectiveness of using this algorithm 
in primary care. 
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Further research is needed to evaluate the clinical outcomes and cost effectiveness of using these algorithms in primary care 
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Tables 



Table | Characteristics of patients at baseline comparing those not taking any anticoagulant, prescribed warfarin, and prescribed another 
type of anticoagulant except for warfarin 




Derivation cohort (n=4 378 851) 


Validation cohort (11=1 358 231) 


Characteristics 


No anticoagulant 


Warfarin 


Other anticoagulant 


No anticoagulant 


Warfarin 


Other anticoagulant 


Female 


22 10 508 (51.1) 


23 740 (45.2) 


841 (51.2) 


687 945 (51 .2) 


6915 (45.5) 


218 (52.4) 


Male 


2 114 217(48.9) 


28 745 (54.8) 


800 (48.8) 


654 679 (48.8) 


8276 (54.5) 


198 (47.6) 


Age group (years): 


21-24 


2 359 220 (54.6) 


3438 (6.6) 


57 (3.5) 


747 255 (55.7) 


1035 (6.8) 


1 1 (2.6) 


45-64 


1 252 978 (29.0) 


11 810 (22.5) 


301 (18.3) 


379 169 (28.2) 


3473 (22.9) 


65 (15.6) 


65-74 


369 120 (8.5) 


13 914(26.5) 


473 (28.8) 


109 989 (8.2) 


4029 (26.5) 


122 (29.3) 


75-84 


233 567 (5.4) 


16 932 (32.3) 


544 (33.2) 


71 127 (5.3) 


4845 (31 .9) 


143 (34.4) 


>85 


109 840 (2.5) 


6391 (12.2) 


266 (16.2) 


35 084 (2.6) 


1809 (11.9) 


75 (18.0) 


Mean (SD) Townsend score 


0.3 (3.6) 


-0.7 (3.2) 


-1.1 (3.0) 


0.5 (3.6) 


-0.6 (3.3) 


-1.2 (3.1) 


Self assigned ethnicity recorded: 


3 200 178 (74.0) 


41 687 (79.4) 


1281 (78.1) 


10 29 330 (76.7) 


12 365 (81.4) 


331 (79.6) 


White or not recorded 


3 819 953 (88.3) 


51 071 (97.3) 


1606 (97.9) 


11 70 896 (87.2) 


14 713 (96.9) 


408 (98.1) 


Indian 


91 005 (2.1) 


277 (0.5) 


9 (0.5) 


28 242 (2.1) 


99 (0.7) 




Pakistani 


45 597 (1.1) 


112 (0.2) 




17681 (1.3) 


43 (0.3) 




Bangladeshi 


35 891 (0.8) 


63 (0.1) 


„ 


11827 (0.9) 


17 (0.1) 




Other Asian 


62 500 (1 .4) 


110 (0.2) 


5 (0.3) 


19131 (1.4) 


28 (0.2) 




Caribbean 


42 109 (1.0) 


326 (0.6) 


6 (0.4) 


15367 (1.1) 


123 (0.8) 




Black African 


79 299 (1 .8) 


153 (0.3) 




30583 (2.3) 


59 (0.4) 




Chinese 


47 733 (1.1) 


42 (0.1) 




11319 (0.8) 


12 (0.1) 




Other 


100 638 (2.3) 


331 (0.6) 


8 (0.5) 


37578 (2.8) 


97 (0.6) 




Smoking status recorded (cigarettes/day): 


4 163 230 (96.3) 


52 405 (99.8) 


1633 (99.5) 


1 294 037 (96.4) 


15 179 (99.9) 


413 (99.3) 


Non-smoker 


2 416 565 (55.9) 


26 673 (50.8) 


852 (51 .9) 


734 433 (54.7) 


7661 (50.4) 


225 (54.1) 


Former smoker 


806 807(18.7) 


20 245 (38.6) 


637 (38.8) 


262 279(19.5) 


5971 (39.3) 


163 (39.2) 


Light smoker (1-9) 


569 657(13.2) 


3339 (6.4) 


84 (5.1) 


180 275 (13.4) 


954 (6.3) 


16 (3.8) 


Moderate smoker (10-19) 


238 430 (5.5) 


1310 (2.5) 


39 (2.4) 


74 830 (5.6) 


350 (2.3) 




Heavy smoker (>20) 


131 771 (3.0) 


838 (1.6) 


21 (1.3) 


42 220 (3.1) 


243 (1 .6) 


7(1.7) 


Alcohol status recorded (units/day): 


3 537 384 (81 .8) 


49 206 (93.8) 


1532 (93.4) 


10 88 990 (81.1) 


14 314(94.2) 


379 (91.1) 


Non-drinker 


11 25 701 (26.0) 


18 388 (35.0) 


524 (31 .9) 


345 074 (25.7) 


5426 (35.7) 


131 (31.5) 


Trivial (<1 ) 


12 03 862 (27.8) 


15 226 (29.0) 


498 (30.3) 


370 619(27.6) 


4339 (28.6) 


120 (28.8) 


Light (1 or 2) 


561 310 (13.0) 


6668 (12.7) 


206 (12.6) 


175 150 (13.0) 


1956 (12.9) 


52 (12.5) 


Moderate (3-6) 


579 526 (13.4) 


7953 (15.2) 


275 (16.8) 


177 676 (13.2) 


2354 (15.5) 


69 (16.6) 


Heavy (7-9) 


42 237 (1 .0) 


681 (1.3) 


21 (1.3) 


12 395 (0.9) 


165 (1.1) 


5(1.2) 


Very heavy (>9) 


24 748 (0.6) 


290 (0.6) 


8 (0.5) 


8076 (0.6) 


74 (0.5) 




Clinical conditions and drugs: 


Previous bleed 


348 785 (8.1) 


12 088 (23.0) 


444 (27.1) 


104 068 (7.8) 


3386 (22.3) 


98 (23.6) 


Oesophageal varices 


3215 (0.1) 


73 (0.1) 




960 (0.1) 


20 (0.1) 




Chronic liver disease/pancreatitis 


20 765 (0.5) 


497 (0.9) 


16 (1.0) 


6656 (0.5) 


156 (1.0) 




Cancer 


118 225 (2.7) 


4561 (8.7) 


123 (7.5) 


36 162 (2.7) 


1282 (8.4) 


41 (9.9) 


Platelets recorded 


21 31 904 (49.3) 


47 435 (90.4) 


1527 (93.1) 


642 051 (47.8) 


13 619 (89.7) 


385 (92.5) 


Most recent platelet <1 50/uL or >480/uL 


66 414(1.5) 


3416 (6.5) 


89 (5.4) 


20 268 (1.5) 


944 (6.2) 


23 (5.5) 


Antiplatelet drug 


314 857 (7.3) 


19 472 (37.1) 


568 (34.6) 


95 597 (7.1) 


5625 (37.0) 


151 (36.3) 


NSAIDS 


701 062 (16.2) 


14 349 (27.3) 


640 (39.0) 


203 539 (15.2) 


4091 (26.9) 


162 (38.9) 


Corticosteroids 


136776 (3.2) 


6009 (1 1 .4) 


162 (9.9) 


39 394 (2.9) 


1812 (11.9) 


41 (9.9) 


Antidepressants 


488 308 (11.3) 


9627 (18.3) 


290 (17.7) 


149 439 (11.1) 


2849 (18.8) 


83 (20.0) 


Phenytoin or carbamazepine 


22 332 (0.5) 


575 (1.1) 


10 (0.6) 


6962 (0.5) 


163 (1.1) 
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Table 1 (continued) 





Derivation cohort (n=4 378 851) 


Validation cohort (n=1 358 231) 


Characteristics 


No anticoagulant 


Warfarin 


Other anticoagulant 


No anticoagulant 


Warfarin 


Other anticoagulant 


Atrial fibrillation 


33 099 (0.8) 


6391 (12.2) 


189 (11.5) 


10 690 (0.8) 


1794 (11.8) 


76 (18.3) 


Venous thromboembolism 


42 072 (1 .0) 


4426 (8.4) 


70 (4.3) 


13 388 (1.0) 


1253 (8.2) 


17(4.1) 


Congestive cardiac failure 


28 071 (0.6) 


2582 (4.9) 


54 (3.3) 


8891 (0.7) 


740 (4.9) 


16 (3.8) 


Treated hypertension 


473 972 (11.0) 


22 507 (42.9) 


685 (41 .7) 


143 734 (10.7) 


6654 (43.8) 


186 (44.7) 


Body mass index recorded 


3 328 834 (77.0) 


47 865 (91.2) 


1519 (92.6) 


1 035 111 (77.1) 


13 855 (91.2) 


377 (90.6) 


Mean (SD) body mass index 


26 (4.8) 


28.4 (5.2) 


28.3 (5.1) 


25.9 (4.8) 


28.5 (5.2) 


28.2 (5.2) 


NSAIDS=non-steroidal anti-inflammatory drugs. 
'Counts <5 have been suppressed. 
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able 2| Incident haemorrhagic events in derivation cohort recorded on mortality or hospital records 



Haemorrhagic events 


Cases on mortality or hospital data Person years Age standardised incidence rate per 1000 person years* (95% CI) 


Upper gastrointestinal bleed: 


No anticoagulants 


21 143 


16 332 684 


1.34 (1.32 to 1.36) 


Warfarin 


492 


71 597 


5.83 (4.33 to 7.33) 


New anticoagulants 


6 


664 


2.69 (0.36 to 5.03) 


Intracranial bleed: 


No anticoagulants 


8 670 


16 332 684 


0.55 (0.54 to 0.56) 


Warfarin 


365 


71 597 


1.89(1.46 to 2.32) 


New anticoagulants 


5 


664 


1.49 (1.27 to 2.86) 


*Age standardised to derivation cohort population. 
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able 3| Adjusted hazard ratios (95% confidence intervals) in derivation cohort for upper gastrointestinal haemorrhagic events recorded 
on mortality or hospital records 



Adjusted hazard ratio*(95% CI) 


Characteristics 


Women 


Men 


Townsend score (5 unit increase) 


1.11 (1.08 to 1.15) 


1.17 (1.13 to 1.20) 


Smoking status (cigarettes/day): 


Non-smoker 1 1 


Former smoker 


1.09 (1.04 to 1.14) 


1.12 (1.07 to 1.18) 


Light smoker (1-9) 


1.31 (1.23 to 1.40) 


1.40 (1.33 to 1.49) 


Moderate smoker (10-19) 


1.30 (1.19 to 1.43) 


1.39 (1.28 to 1.52) 


Heavy smoker (>20) 


1 .56 (1 .39 to 1 .74) 


1.62(1.48 to 1.76) 


Ethnicity: 


White or ethnicity not recorded 1 1 


Indian 


0.96 (0.80 to 1.16) 


0.97 (0.82 to 1.14) 


Pakistani 


0.99 (0.76 to 1.28) 


0.82 (0.65 to 1.04) 


Bangladeshi 


0.88 (0.65 to 1.19) 


0.94 (0.74 to 1.19) 


Other Asian 


0.81 (0.62 to 1.06) 


0.95 (0.75 to 1.19) 


Caribbean 


1.14 (0.95 to 1.36) 


1.11 (0.92 to 1.33) 


Black African 


0.54 (0.41 to 0.72) 


0.85 (0.69 to 1.05) 


Chinese 


1 .05 (0.77 to 1 .44) 


0.91 (0.66 to 1 .26) 


Other 


1 .04 (0.87 to 1 .24) 


0.94 (0.79 to 1.11) 


Alcohol intake (units/day): 


None 1 1 


Trivial (<1 ) 


0.87 (0.83 to 0.92) 


0.82 (0.78 to 0.86) 


Light (1-2) 


0.79 (0.74 to 0.85) 


0.82 (0.77 to 0.87) 


Moderate (3-6) 


0.99 (0.92 to 1.07) 


0.89 (0.85 to 0.95) 


Heavy (7-9) 


1.85 (1.47 to 2.32) 


1.35 (1.21 to 1.50) 


Very heavy (>9) 


2.85 (2.27 to 3.59) 


1.79 (1.57 to 2.04) 


Clinical conditions"! - : 


Previous bleed 


2.26 (2.10 to 2.43) 


2.11 (1.98 to 2.26) 


Oesophageal varices 


3.35 (2.58 to 4.33) 


2.07 (1 .59 to 2.68) 


Chronic liver disease/pancreatitis 


2.81 (2.44 to 3.23) 


3.08 (2.75 to 3.45) 


Atrial fibrillation 


1.24(1.11 to 1.39) 


1.21 (1.07 to 1.36) 


Venous thromboembolism 


1.16 (1.04 to 1.30) 


1.12 (0.98 to 1.28) 


Congestive cardiac failure 


1.41 (1.25 to 1.59) 


1.46 (1.30 to 1.64) 


Treated hypertension 


1.06(1.01 to 1.12) 


1.11 (1.05 to 1.16) 


Cancer 


1.22 (1.13 to 1.31) 


1.39 (1.29 to 1.50) 


Most recent platelet count <150/uL or >480/uL 


1 .68 (1 .52 to 1 .85) 


1.79 (1.64 to 1.96) 


Drugst: 


Anticoagulant 


3.89 (2.75 to 5.49) 


4.43 (3.32 to 5.92) 


Antiplatelet drug 


1.26 (1.19 to 1.33) 


1.21 (1.1 5 to 1.28) 


NSAIDS 


1.16 (1.11 to 1.21) 


1.09 (1.05 to 1.14) 


Corticosteroids 


1.26 (1.18 to 1.34) 


1.26 (1.1 7 to 1.36) 


Antidepressants 


1.57 (1.51 to 1.64) 


1.69 (1.60 to 1.78) 


Phenytoin or carbamazepine 


1.40 (1.18 to 1.65) 


1.30 (1.10 to 1.55) 



NSAIDS=non-steroidal anti-inflammatory drugs. 

Models also included fractional polynomial terms for age and body mass index. 

Fractional polynomial terms for women were (age/1 Of' and (age/1 0) 3 ; (body mass index/1 0)~ 2 and (body mass index/1 0)~ z ln(body mass index). Fractional polynomial 
terms for men were (age/10) ~' and (age/10) 3 In(age); (body mass index/10)~ z and (body mass index/10)~ z ln(body mass index). 

The models for men and women also included interactions between the age terms and anticoagulant use and between age and previous bleed. Hazard ratios for 
these variables in the table are evaluated at mean age in men and women. 
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Table 3 (continued) 







Adjusted hazard ratio*(95% CI) 




Characteristics 


Women 




Men 



'Hazard ratios simultaneously adjusted for all other variables shown in table as well as for fractional polynomial terms for age and body mass index. 
tCompared with patients without condition or drug at baseline. 
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able 4| Adjusted hazard ratios (95% confidence interval) in the derivation cohort for intracranial haemorrhagic events recorded on mortality 
or hospital records 

Adjusted hazard ratio* (95% CI) 



Characteristics 


Women 


Men 


Townsend score (5 unit increase) 


1.15 (1.10 to 1.20) 


1.12 (1.07 to 1.17) 


Smoking status (cigarettes/day): 


Non-smoker 1 1 


Former smoker 


1.12 (1.04 to 1.21) 


1.04 (0.97 to 1.12) 


Light smoker (1-9) 


1.80 (1.63 to 1.99) 


1.44 (1.31 to 1.58) 


Moderate smoker (10-19) 


2.12(1.86 to 2.43) 


1.72 (1.50 to 1.97) 


Heavy smoker (>20) 


2.37 (2.01 to 2.81) 


1.72(1.47 to 2.00) 


Ethnicity: 


White or not recorded 1 1 


Indian 


1.02 (0.73 to 1.42) 


1.08 (0.82 to 1.42) 


Pakistani 


0.92 (0.52 to 1.63) 


1.02 (0.69 to 1.52) 


Bangladeshi 


1.60 (1.00 to 2.56) 


1.15 (0.76 to 1.73) 


Other Asian 


0.94 (0.58 to 1.52) 


1.44(1.03 to 2.03) 


Caribbean 


1.14(0.84 to 1.56) 


1.32 (1.01 to 1.72) 


Black African 


1.20 (0.80 to 1.80) 


1.56 (1.16 to 2.11) 


Chinese 


1.43 (0.84 to 2.42) 


1.15 (0.66 to 1.98) 


Other 


1.14 (0.83 to 1.56) 


1.02 (0.75 to 1.38) 


Alcohol intake (units/day): 


None 1 1 


Trivial (<1 ) 


0.96 (0.89 to 1.03) 


0.84 (0.77 to 0.92) 


Light (1-2) 


1.03 (0.92 to 1.16) 


0.85 (0.77 to 0.93) 


Moderate (3-6) 


1.05 (0.93 to 1.18) 


0.97 (0.88 to 1.06) 


Heavy (7-9) 


2.13(1.49 to 3.03) 


1.58 (1.34 to 1.86) 


Very heavy (>9) 


2.62 (1 .73 to 3.97) 


1.48 (1.14 to 1.92) 


Clinical conditions"! - : 


Previous bleed 


1.33 (1.23 to 1.44) 


1.32 (1.23 to 1.42) 


Oesophageal varices 


3.22 (2.07 to 5.01) 


1.72(1.01 to 2.91) 


Chronic liver disease or pancreatitis 


1.92 (1.50 to 2.46) 


2.21 (1.79 to 2.73) 


Atrial fibrillation 


1.17 (1.01 to 1.35) 


1.36 (1.18 to 1.57) 


Treated hypertension 


1.06 (1.00 to 1.14) 


1.18 (1.09 to 1.26) 


Most recent platelet count <150/uL or >480/uL 


1.62 (1.41 to 1.88) 


1.61 (1.41 to 1.83) 


Drugst: 


Anticoagulant 


3.62 (1.25 to 10.49) 


3.99 (1.86 to 8.55) 


Antiplatelet drug 


1.31 (1.22 to 1.41) 


1.27 (1.17 to 1.36) 


Antidepressants 


1.31 (1.22 to 1.40) 


1.38 (1.27 to 1.50) 


Phenytoin or carbamazepine 


2.20 (1 .79 to 2.70) 


2.03 (1.64 to 2.53) 



NSAIDS=non-steroidal anti-inflammatory drugs. 

Models also included fractional polynomial terms for age and body mass index. 

Fractional polynomial terms for women were (age/1 Of' and (age/1 0) 3 ; (body mass index/1 0)~ 2 and (body mass index/1 0)~ z ln(body mass index). Fractional polynomial 
terms for men were (age/10) ~' and (age/10) 3 In(age); (body mass index/10)~ z and (body mass index/10)~ z ln(body mass index). 

The models for men and women also included interactions between the age terms and anticoagulant use. Hazard ratios for these variables in the table are evaluated 
at mean age in men and women. 

*Hazard ratios simultaneously adjusted for all other variables shown in table as well as for fractional polynomial terms for age and body mass index. 
tCompared with patients without condition or drug at baseline. 
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Table 5| Performance of each algorithm In validation cohort using incident events included in hospital or mortality data 







Mean (95% CI) 


Incident events 


Women 


Men 


Upper gastrointestinal bleed: 


ROC statistic* 


0.766 (0.758 to 0.775) 


0.747 (0.738 to 0.756) 


R 2 (%)t 


40.7 (38.9 to 42.6) 


36.9 (35.1 to 38.7) 


D statistic}: 


1.70(1.63 to 1.76) 


1.57 (1.51 to 1.63) 


Intracranial bleed: 


ROC statistic* 


0.847 (0.838 to 0.856) 


0.812 (0.80 to 0.824) 


R 2 (%)t 


58.0 (56.0 to 60.0) 


53.3 (51.1 to 55.4) 


D statistic} 


2.40 (2.30 to 2.50) 


2.19(2.09 to 2.28) 



Discrimination is ability of risk prediction model to differentiate between patients who do and do not experience an event during the study. 
'Receiver operating characteristic (ROC) curve statistic; higher values indicate better discrimination. 
tMeasures explained variation, with higher values indicating more variation explained. 

^Measure of discrimination specific to censored survival data. As with ROC, higher values indicate better discrimination. 
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Table | Performance of each model In the validation cohort based on top 10% of patients at highest risk (n=1 35 823 patients). Values are 


point estimates (95% confidence intervals) 








Events 


Cut-off 5 year 
risk (%) 


Sensitivity (%) Specificity (%) 


Negative predictive 
value (%) 


Positive likelihood 
ratio 


Observed risk* at 5 
years (%) 


Upper 

gastrointestinal 
bleed 


1.4 


38.0(36.81 to 39.17) 90.14(90.09to90.19) 


99.67 (99.65 to 99.68) 


3.85 (3.83 to 3.97) 


2.70 (2.60 to 2.80) 


Intracranial bleed 


0.7 


51 .03 (49.1 7 to 52.89) 90.09 (90.03 to 90.14) 


99.89 (99.88 to 99.89) 


5.15 (4.96 to 5.34) 


1.50 (1.45 to 1.62) 


'Estimate of positive predictive value. 
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Fig 1 Graphs showing fractional polynomial terms for age (hazard ratios compared with age 40) and body mass index 
(hazard ratios compared with body mass index 25) and age interactions for upper gastrointestinal bleed 
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Fig 2 Graphs showing fractional polynomial terms for age (hazard ratios compared with age 40) and body mass index 
(hazard ratios compared with body mass index 25) and interaction between anticoagulant use and age for intracranial bleed 
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Fig 3 Mean predicted risks and observed risks at five years by 1 0th of predicted risk applying QBleed risk prediction scores 
to all patients in QResearch validation cohort 
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Example of 70 year old female 

About you 

Age (21-99): 1 70 \ 
5ex: O Male ® Female 

Ethnicity: | White or not stated t | 
UK postcode: leave blank if unknown 
Postcode: | ~| 



Clinical information " 



Smoking status: | moderate smoker (10 to 19) t] 
Alcohol status: | >9 units per day t"| 
Do you have... 



atrial fibrillation? 0 

heart failure? □ 

high blood pressure needing medication? □ 

a current or previous diagnosis of cancer? □ 
chronic liver disease or chronic pancreatitis? □ 

oesophageal varices? □ 

a platelet count <1 50 ug/L or >480 ug/L? □ 



Have you had... 

previous bleed (gastrointestinal, intracranial, □ 

haematuria, haemoptysis)? 

venous thromboembolism? □ 



Are you currently taking... 

antiplatelet drugs (e.g. aspirin, clopidogrel □ 
or dipyramidole)? 

non-steroidal anti-inflammatory pain killers 0 
(NSAIDS)? 

regular steroid tablets? 0 

antidepressants? □ 

anticonvulsants (phenytoin orcarbamezepine)? □ 



Leave blank if unknown ■ 

Body mass index 

Height (cm): 1 160 
Weight (kg): 



| 80 



H 



Calculate risk over | 1 ▼ | years 

Example of 80 year old male 

About you 

Age (21-99): 1 80 | 
Sex: ® Male O Female 

Ethnicity: | White or not stated t~| 
UK postcode: leave blank if unknown 
Postcode: |~~ 



Calculate risk 



Clinical information " 



Smoking status: | moderate smoker (10 to 19) t 
Alcohol status: | >9 units per day * 

Do you have... 

atrial fibrillation? □ 
heart failure? 0 
high blood pressure needing medication? □ 
a current or previous diagnosis of cancer? □ 
chronic liver disease or chronic pancreatitis? □ 
oesophageal varices? □ 
a platelet count <1 50 ug/Lor >480 ug/L? 0 



Have you had... 

previous bleed (gastrointestinal, intracranial, 0 

haematuria, haemoptysis)? 

venous thromboembolism? □ 



Are you currently taking... 

antiplatelet drugs (e.g. aspirin, clopidogrel 0 
or dipyramidole)? 

non-steroidal anti-inflammatory pain killers 0 
(NSAIDS)? 

regular steroid tablets? 0 

antidepressants? □ 

anticonvulsants (phenytoin orcarbamezepine)? □ 



Leave blank if unknown ■ 

Body mass index 

Height (cm): 1 160 
Weight (kg): | 60 



Your risk of having a serious bleed requiring hospital admission or causing 
death within the next year is: 





Before anticoagulant | 


On anticoagulant 


| Upper gastrointestinal bleed 


o.« |! 


1.7% 


| Intracranial bleed 


0.3% || 


1.3% 



Your risk of having a serious bleed requiring hospital admission or causing 
death within the next year is: 





Before anticoagulant | 


On anticoagulant 


| Upper gastrointestinal bleed 


2.9% 


6% 


| Intracranial bleed 


1.1% || 


3% 



Calculate risk over | 1 t | \ 



| Calculate risk] 

Fig 4 Example of risk of bleed with and without anticoagulants in two patients using the web calculator at www.qbleed.org 
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